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Abstract. The purpose of this note is to survey a methodology to solve systems 
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1. Introduction. A wide variety of problems in optimization can be 
easily modeled using systems of polynomial equations and inequalities. Fea- 
sibility and optimization problems translate, either directly or via branch- 
ing, into the problem of finding a solution of a system of equations and 
inequalities. In this survey paper, we explain how to manipulate such sys- 
tems for finding solutions or proving that they do not exist. Although these 
techniques work in general, we are particularly motivated by problems of 
combinatorial origin. For example, in the case of graphs, here is how one 
can think about stable sets, A:-colorability and max-cut problems in terms 
of polynomial (non-linear) constraints: 

Proposition 1.1. Let G — {V,E) he a graph. 
• For a given positive integer k, consider the following polynomial 
system: 



Vi S V, XiXj = y{i,j) G E and Xi = k. 



This system is feasible if and only if G has a stable set of size k. 
For a positive integer k, consider the following polynomial system 
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o/ 1^1 + l^"! polynomials equations: 



k-l 



1 = Vi e y and x\ 



,k — l — ss 



x]^Qy[i,])eE. 



s=0 



The graph G is k-colorahle if and only if this system has a complex 
solution. Furthermore, when k is odd, G is k-colorable if and only 
if this system has a common root over ¥2, the algebraic closure of 
the finite field with two elements. 
• We can represent the set of cuts of G (i.e., bipartitions onV) as 
the 0-1 incidence vectors 

SG --^{x^ ■■ F <ZE is contained in a cut of G} C {0, 1}^. 

Thus, the max cut problem with non-negative weights We on the 
edges e ^ E is 



There are many other combinatorial problems that can be modeled 
concisely by polynomial systems (see and the many references therein) . 
In fact, a given problem can often be modeled non-linear ly in many differ- 
ent ways, and in practice choosing a "good" formulation is critical for an 
efficient solution. 

Given a polynomial system encoding a combinatorial question, we ex- 
plain how to use two famous algebraic identities to derive solution methods. 
In what follows, let K denote a field and let K denote the algebraic clo- 
sure of K. Let R = K[xi, . . . ,Xn] — K[x] denote the ring of polynomials 
in n variables with coefficients over K. The situation is slightly different 
depending on whether only equations arc being considered, or if there also 
inequalities (more precisely, on whether the underlying field K[x] is alge- 
braically closed or formally real): 

1. First, suppose that the system contains only the polynomial equa- 



tions fi{x) = 0, f2{x) = 0, . . . , fs{x) = where /i, fs £ K[x 



We explain how to generate a finite sequence of linear algebra sys- 
tems which terminate with either a solution over IK of the problem 
or provide a certificate of infeasibility. The calculations reduce to 
matrix manipulations, mostly rank computations. The techniques 
we use are a specialization of prior techniques from computational 
algebra (see [36llini|2Tl|37]). As it turns out this technique is par- 
ticularly effective when the number of solutions is finite, when K 



max{ 




The vectors x^ a^^e the solutions of the polynomial system 
xl — x^ — OyeeE, and = V T an odd cycle in G. 
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is a finite field, or when tlie system has nice combinatorial infor- 
mation (see H]). 

2. Second, several authors (see e.g. [l^lHnilMj and references therein) 
considered the solvability (over the reals) of systems of polynomial 
equations and inequalities. It was shown that in this situation 
there is a way to set up the feasibility problem 

3x e R" s.t. /i(a;) = 0, . . . , Mx) = 0, 51(2;) > 0, ... , gk{x) > 0, 

where fi, . . . , fs,gi, ■ ■ ■ ,gk G M.[x\, as a sequence of semidefinite 
programs terminating with a feasible solution (see [IH]). Once 
more, the combinatorial structure can help in the understanding 
of the structure of these relaxations, as is well-known from the case 
of stable sets [31] and max-cut [27] . In recent work, Gouveia et al. 
[T5l[T4] considered a sequence of semidefinite relaxations of the con- 
vex hull of real solutions to an arbitrary combinatorial polynomial 
system. They called these approximations theta bodies because for 
stable sets of graphs the first theta body in this hierarchy is exactly 
Lovasz's theta body of a graph 131]. 
The common central idea to both of the relaxations procedures de- 
scribed above is to use the right infeasibility certificates or theorems of 
alternative. Just as Farkas' lemma is a centerpiece for the development of 
Linear Programming, here the key point is that the infeasibility of poly- 
nomial systems can always be certified by particular algebraic identities 
(on non- linear polynomials). To find these infeasibility certificates we rely 
either on linear algebra or semidefinite programming (for a quick overview 
of semidefinite programming see [49j ) . 

We now state the necessary notation and algebraic concepts that jus- 
tify our approach. For a detailed introduction we recommend the books 
m 121 [35]. We denote the monomials in the polynomial ring R = 
K[a;i,...,a;„] = K[x] as x"^^ x^^ ■ ■ ■ x^" for a £ W\ The degree 

of a:" is deg(x") :— \a\ :— X^iLi*^*- "^^"^ degree of a polynomial / = 
SqsN" faX°'i written deg(/), is the maximum degree of a;" where fa^O 
for a € N". Given a set of polynomials F C R, we write deg(F) for the 
maximum degree of the polynomials in F. The variety of F over K, writ- 
ten Vk(^"), is the set of common zeros of polynomials in F in K", that is, 
Vk{F) := {^; e K" : f{v) = V/ G /}. Also, V^{F), the variety of F 
over K, is the set of common zeros of in K . Note that in combinatorial 
problems, the variety of a polynomial system typically has finitely many 
solutions (e.g., colorings, cuts, stable sets, etc.). 

Given a set of polynomials F {/i, . . . , f„i} C _R = K[x], we define 
the ideal of F as 

{F)r:^ {f^,...Jrn)R:^ l^A/, |/3i,...,/3™eK[x]|. 
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For an ideal I ^ R, when V-^{I) is finite, the ideal is called zero- dimensional 
(this is the case for all of the applications considered here). An ideal ICR 
is radical if /^^ G / for some positive integer k implies f G I- We denote by 
\/T the ideal of all polynomials f & R such that /''" e I for some positive 
integer k. The ideal vT is necessarily radical and it is called the radical 
ideal of /. Note that / is radical if and only if / = vT. 

To study of varieties over a non-algebraically closed field like M requires 
extra structure. Given a set of real polynomials G :— {gi, . . . , g,n} Q IR.[a:], 
we define the cone of G as 

cone(G) := | g = so + ^ s^g^ + ^ Si^g^g^ + ^ Sijkgzgjgk + ■■■}, 

where each term in the sum is a square-free product of the polynomials gi, 
with a coefhcient Sa S M.[x] that is a sums of squares. The sum is finite, 
with a total of 2™ — 1 terms, corresponding to the nonempty subsets of 
{51, ■ • ■ ,5m}- 

The notions of ideal and cone are standard in real algebraic geom- 
etry, but they also have inherent convex geometry: Ideals are affine sets 
and cones are closed under convex combinations and non-negative scal- 
ings, i.e., they are actually cones in the convex geometry sense. Ideals 
and cones are used for deriving new valid constraints, which are logical 
consequences of the given constraints. For example, notice that by con- 
struction, every polynomial in (/i, . . . , fm)R vanishes in the solution set 
of the system /i(x) = 0, . . . , fm{x) — over the algebraic closure of K. 
Similarly, every element of cone(gi) is clearly non-negative on the feasible 
set of gi{x) > 0, . . ■,gm{x) > 0. 

It is well-known that optimization algorithms are intimately tied to the 
development of feasibility certificates. For example, the simplex method is 
closely related to Farkas' lemma. Our starting point is a generalization of 
this famous principle. We start with a description of two powerful infea- 
sibility certificates for polynomial systems which generalizes the classical 
ones for linear optimization. First, recall from elementary linear algebra 
the "Fredholm alternative theorem" (e.g., [44]): 

Theorem 1.1 (Fredholm's alternative). Given a matrix A e K™^" 
and a vector b € K™, 

$xeK" s.t. Ax + b = ^ 3 fie K" s.t. fF A ^ 0, fFh = 1. 

It turns out that there are much stronger versions for general polynomials, 
which unfortunately does not seem to be widely known among optimizers 
(for more details see e.g., [5]). 

Theorem 1.2 (Hilbert's NuUstellensatz). Let F {/i,...,/„J C 
K.[x] . Then, 

$xeK" s.t. h{x)^o,...,Mx)^O^ie {f)r. 
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Note that 1 e (F) ^ means that there exist polynomials Pi, . . . ,f3m G 
K[a;] such that 1 = X]"=i A/i- Note that Fredholm's alternative theorem is 
simply a special case of Hilbert's NuUstellensatz where all the polynomials 
are linear and the /3i's are constant. 

Now, the two theorems above deal only with the case of equations. 
The inclusion of inequalities in the problem formulation poses additional 
algebraic challenges because we need to take into account special properties 
of the reals. Consider first the case of linear inequalities where linear 
programming duality provides the following characterization: 

Theorem 1.3 (Farkas' lemma). Let A e M"^", b G M™, C G M''^", 
and d e R'=. 

$x eM." s.t. Ax + b = 0,Cx + d>0 

t 

3 A e M™, 3 e M*^ s.t. ^FA + A^C = 0, ^Fb + X^d = -1. 

Again, although not widely known in optimization, it turns out that sim- 
ilar certificates do exist for arbitrary systems of polynomial equations and 
inequalities over the reals. The result essentially appears in this form in 
[2], and is due to Stengle f47l. 

Theorem 1.4 (Positivstellensatz) . Let F := {/i, . . . , /„} c M.[x] and 
G :={<?!,..., 5fc}cMH. 

h e M" s.t. = 0, . . . , f,n{x) = 0,gi{x) >0,...,gk{x)>0 

t 

3/e (F)^,3.gecone(G) s.t. f{x)+g{x) = ~l 

The theorem states that for every infeasible system of polynomial equa- 
tions and inequalities, there exists a simple algebraic identity that directly 
certifies the non-existence of real solutions. 

Of course, we are very concerned with the effective practical compu- 
tation of the infeasibility certificates. For the sake of computation and 
complexity, we must worry about the growth of degrees of the infeasibility 
certificates. On the negative side, the degrees of the certificates are ex- 
pected to be high (in the worst case) simply because the NP-hardness of 
the original combinatorial questions; see e.g. 9 . At the same time, tight 
exponential upper bounds have been derived (see e.g. [55], [TB] and refer- 
ences therein). Nevertheless, for many problems of practical interest, it is 
often the case that it is possible to prove infeasibility using low-degree cer- 
tificates (see 13 E]). Even more important is the fact that for fixed degree 
of the certificates the calculations can be reduced to either linear algebra 
or semidefinite programming. We summarize the strong analogies between 
the case of linear equations and inequalities with high-degree polynomial 
systems in the following table: 
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Degree\Field 


Arbitrary 


Real 


Linear 


Fredholm Alternative 


Farkas ' Lemma 




Linear Algebra 


Linear Programming 


Polynomial 


Nullstellensatz 


Positivstellensatz 




Bounded degree Linear Algebra 


Bounded degree SDP 




Table 1 





Infeasibility certificates and their associated computational techniques. 



It is important to remark that just as in the classical case of linear 
programming, the problem of computation of certificates has very natural 
primal-dual formulations, with the corresponding primal and dual vari- 
ables playing distinct, but well-define roles. For example, in the case of 
Fredholm's alternative, the primal variables are the variables xi,...,Xn 
while there is a dual variable for each equation. For Nullstellensatz and 
Positivstellensatz there is a similar duality, based on linear duality and 
semidefinite programming duality, respectively. In what follows, we use 
the most intuitive or convenient set-up and we leave the reader the exer- 
cise of transferring the results to the corresponding dual version. 

The remainder of the paper is divided in two main sections: Section 
[5] is a study of the Hilbert Nullstellensatz, for general fields, used in the 
solution of systems of equations. In Section [31 we survey the use of the 
Positivstellensatz in the context of solving systems of equations and in- 
equalities over the reals. Both sections contain combinatorial applications 
that show why these techniques can be of interest in this setting. The fo- 
cus of the combinatorial results is understanding those situations when a 
constant degree certificate is enough to show infeasibility. These are situa- 
tions when hard combinatorial problems have polynomial time algorithms 
and as such provide structural insight. Finally, in Section |4l we describe a 
methodology, common to both approaches, to recover feasible solutions of 
the original combinatorial problem from the outcome of these relaxations. 

To conclude the introduction we include some more notation. Given a 
vector space W over a field K, we write dim(iy) for the dimension of W. 
Given vector spaces U C W, we write W/U as the vector space quotient. 
Recall that dim{W/U) ~ dim(T4^) — dim(C/). As a slight abuse of notation, 
if t/ 2 W, then we write W/U when we strictly mean W/{U n W), in 
which case, dim{W/U) = dim{W) - dim{U nW) . Given a set F C R, (F)k 
denotes the vector space generated by F over the field K. Please note the 
distinction between the vector space (F)^ and the ideal (F)]^. 

2. Solving combinatorial systems of equations. In this section, 
we wish to solve a given zero-dimensional system of polynomial equations 
fi{x) = 0,f2{x) = 0, . . . , fm{x) = where /i, . . . , /s € i?. We abbreviate 
this system as F{x) = where F := {/i, . . . , /„} C R. Here, by solving 
a system, we mean first determining if F{x) = is feasible over K, the 



POLYNOMIALS IN COMBINATORIAL OPTIMIZATION 



7 



algebraic closure of K, and furthermore finding a solution (or all solutions) 
of F{x) = if feasible. We say that a system is combinatorial when it 
is defined in terms of combinatorial information such as graph properties 
and it has finitely many solutions (if any). The literature on polynomial 
solving is very extensive and it continues to be an area of active research 
(see [m m [To] for an overview and background) . 

Here we choose to focus on techniques that fit well with optimization 
methods. The main idea is that solving a polynomial system of equations 
can be reduced to solving a sequence of linear algebra problems. The foun- 
dations of this technique can be traced back to ( [36l [20l [211 [37] ) . Variants 
of this technique have been applied to stable sets [9l [34] , vertex coloring 
[8l[34], satisfiability (see e.g., [3]) and cryptography (see for example [4]). 
This technique is also strongly related to Grobner bases techniques (see 
e.g., [1D1[371[1H|). 

The linear algebra systems of equations have primal and dual represen- 
tations in the sense of Fredholm's lemma. Specifically, in this survey, the 
primal approach solves a linear system to find constant multipliers fi € K™ 
such that 1 = providing a certificate of (non-linear) infeasibil- 

ity. Then, the dual approach aims to find a vector A with entries in IK 
indexed by monomials such that J2a ^x^fi.a = for alH = 1, . . . , m and 
Ai = 1 where fi = J2a fi,aX°' for all i. As we see in Section [2?2] the dual 
approach amounts to constructing linear relaxations of the set of feasible 
solutions. In Sections 12. f I and 12.21 we present examples of the primal and 
dual approaches respectively. 

2.1. Linear algebra certificates. Consider the following consequence| 

of Hubert's NuUstcUcnsatz: If there exists constants fjL S K™ such that 
J2iLif^ifi — 1; then the polynomial system F{x) = must be infeasi- 
ble. In other words, if the system F{x) = is infeasible, then 1 S (-F')k- 
The crucial point here is that determining whether there exists a fi Cz K™ 
such that X]"=i A'i/i = 1 is a linear algebra problem over K. The equa- 
tion X)"=i ^^ifi ~ 1 called a certificate of infeasibility of the polynomial 
system. 

Example 1. Consider the following infeasible system in M.[xi, X2, x^]: 

— 1 = 0, 2xiX2 + X3 ^ 0, Xi + X2 = 0, Xi + X3 = 0. 

Let F = {/i,/2,/3,/4} where = - 1 = 0, /2 = 2xiX2 + X3 = 0, 
fs — xi + X2 = 0, and /4 = +X3 = 0. So, we abbreviate the above system 
as F(x) = 0. We can prove that the system F{x) ^ is infeasible if we 
can find /z € satisfying the following: 

+ M2/2 + fJ-sfs + IHfi = 1 
~ 1) + M2(2a;i.T2 + 2:3) + l^zixi + X2) + ^4(2^1 + 2^3) = 1 
<^ Hixl + 2^2a;ia;2 + (^^2 + ^4)2:3 + /i3a;2 + (^3 + Ih)xi - /ii = 1. 
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Then, equating coefficients on the left and right hand sides of the equation 
above gives the following linear system of equations: 

-/il = 1 (1), /i3 + /X4 = (xi), Ms = (2:2), 

A*3 + /M = (xs), 2^2 = [X1X2), A^i = (a^i). 

We abbreviate this system as jf^ F — 1. Even though F{x) = is infeasible, 
the linear system fi^F = 1 is infeasible, and so, we have not found a 
certificate of infeasibility. 

More formally, let fi — J2aGN^ fi-a^" ^o^' * ~ l,...,m. Note that 
only finitely many fi^a are non-zero. Then, l^ifi = 1 if and only if 

S"=i Mi/i,o = 1 and J2^iLi l^ifi,a = for all a £ N" where a ^ 0. Note that 
there is one linear equation per monomial appearing in F. We abbreviate 
this linear system as ^"'^ F = 1 where we consider _F as a matrix whose rows 
are the coefficient vectors of its polynomials and we consider the constant 
polynomial 1 as the vector of its coefficients (i.e., a unit vector). The 
columns of F are indexed by monomials with non-zero coefficients. 

We remark that in the special case where F{x) = is a linear system 
of equations, then Fredholm's alternative says that F{x) = is infeasible 
if and only if fi^F = 1 is feasible. 

In general, even if F{x) = is infeasible, ff^F — 1 may not be feasible 
as in the above example. In order to prove infeasibility, we must add 
polynomials from {F)ji to F and try again to find a /i such that /i-^F = 1. 
Hubert's NuUstellensatz guarantees that, if F{x) = is infeasible, there 
exists a finite set of polynomials from {F)ii that we can add to F so that 
the linear system js^F = 1 is feasible. 

More precisely, it is enough to add polynomials of the form x^f for 
x" a monomial and some polynomial f d F. Why is this? If F{x) = 
is infeasible, then Hilbert's NuUstellensatz says J27=i f^ifi ~ ^ ^'^^ some 
Pi, . . . , & R- Let d = maxi{deg(/3i)}. Then, if we add to F all polyno- 
mials of the form x" f where f G F and deg(a;") < d. Then, the K- linear 
span of F, that is (-F)k, contains (3ifi for all i, and thus, 1 G {F)k or 
equivalently fi^F' = 1 is feasible (as a linear algebra problem) where F' 
denotes the larger polynomial system. 

Example 2. Consider again the polynomial system F{x) — from Ex- 
ample[l\ Here, ji^ F — 1 is feasible, so we must thus add redundant polyno- 
mial equations to the system F{x) = 0. In particular, we add the following 
redundant polynomial equations: X2fi{x) — 0, Xif2{x) = Q, Xif^{x) — 0, 
and xifiix) = 0. Let F' {/i, /2, /s, A, 2:2/1, 2:1/2, 2^1/3, a;i/4}. 

Then, the system ^'^ F' = 1 is now as follows: 

-Hl = l (1), ^3 + /i4 = (xi), /i3 - /is (0:2), 

^J■2 + ^J■i = o (xs), 2/i2 + M7 = o (X1X2), ^Il + ^l7 + ^J■& = o (xl), 

A*6 + Ai8 = {X1X3), /i5-t-2/i6 = {xlx2). 
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This system is feasible proving that Fix) = is infeasible. The solution is 
^ ~ {—1, — |, — |, |, — |, — ^, |, —5)7 which gives the following certificate of 
infeasibility: 

2 2 2 2 1 4 1 

-fi - g/2 - g/3 + - ■^X2fi + -xif2 + -xifs - -a;i/4 = 1. 

Next, we present the dual approach to the one in this section. 

2.2. Linear algebra relaxations. In optimization, it is quite com- 
mon to "linearize" non-linear polynomial systems of equations by replacing 
all monomials in the system with new variables giving a system of linear 
constraints. Specifically, we can construct a linear algebra relaxation of 
the solutions of F(x) = by replacing every monomial in a polynomial 
equation in F{x) = with a new variable A^^ thereby giving a system 
of linear equations in the new A variables, one variable for each mono- 
mial appearing in F. Readers familiar with relaxation procedures such as 
Sherali-Adams and Lovasz-Schrijver (see [55] and references therein) will 
see a lot of similarities, but here we deal only with equality constraints. 

Example 3. Consider the following feasible system in 0:2, X3] ; 

fi{x) ^ xj - 1 = 0, f2{x) ^ 2xiX2 + X3 ^ 0, faix) ^ xi + X2 = 0. 

This system has two solutions {xi, X2, X3) = (1,-1,2) and (xi, X2, X3) = 
(—1,1,2). Let F — {/i,/2,/3}- So, we abbreviate the above system as 
F{x) — 0. We can replace the monomials 1, xi, a;2, 2:3, a;^, X1X2 with the 
variables Xi, X^^^, Xx2i ^xa, X^j, Xxj^x2 respectively. The system F{x) = 
thus gives rise to the following set of linear equations: 

X,2 - Ai = 0, 2A,,,, + A,3 = 0, A,, + A,, = 0. (2.1) 

We abbreviate the above system as F * X = 0. 

Solutions of Fix) = give solutions of F *X — Q: If x is a solution of 
F{x) — above, then setting Ai — l,Xxi — ^11^x2 = X2,Xx3 — X3,X^2 — 
Xi, XxiX2 — X1X2 gives a solution of F*X = 0. 5*0, taking x = (1, —1, 1), we 
set Ai = 1, Xxi = I, Xx2 = -1, Xxa = 2, Aj,2 = 1, and XxiX2 = -1- Then, 
we have F ^ X = 0. Thus, the solutions of F * X = gives a vector space 
effectively containing all of the solutions of F{x) = 0. Hence, F X = 
gives a linear relaxation of F{x) = 0. 

There are solutions of F * X — Q that do not correspond to solutions 
of F(x) — because the linear system _F * A = does not take into account 
the non-linear constraints that Ai = 1, X^2 — A^^ and XxiX2 = ^xiXx2; For 
example, Ai = 1, Xxi — 2, Xx2 = —2, Xx^ — —2, A3.2 = 1 and XxiX2 = 1 is a 
solution of F*X ~ 0, butxi = Xx^ =2, X2 = Xx2 = —2, andx^ — Xx^ — —2 
is not a solution of F{x) = 0. 

We now formalize the above example construction of a linear system. 
We can consider the polynomial ring R — Klxi, . . . , Xn] as an infinite di- 
mensional vector space over K where the set of all monomials forms a 
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vector space basis of R. In other words, a polynomial / = X^qgN" /a^^" 
can be represented as an infinite sequence {fa)a&i" where only finitely 
many are non-zero. We define R* = K[[a;i, . . . , a:;„]] = K[[a;]] as the 
ring of formal power series in the variables x\,...,Xn with coefficients in 
K. So, the power series A — X^qgN" ^aX°' can be represented as an in- 
finite sequence (AajaeN"- Note that we do not require that only finitely 
many Aa are non-zero. We define the bilinear form * : i? x .R* — > K as 
follows: given / = Z"-^" ■= ^^id A ~ Z^aeN- ^aX°' & R*,wc define 

/ * A = X]Q,£f}n /aAa, which is always finite since only finitely many are 
non-zero. Thus, we define a linear relaxation of x S K", F{x) — 0, written 
as A G -R*, * A = 0, as the set of linear equations / * A = for all / G F. 

Note that, for any polynomial f £ R and any point v G K", we have 
f{v) — f *\v where A^ = (f Thus, for any G K , F{v) = if and 

only if F * At, = 0. So, the system F * A = can be considered as a linear 
relaxation of the system F{x) = 0. As mentioned in the above example, 
there are solutions of F * A = that do not correspond to solutions of 
F(x) = because the linear system F * A = does not take into account 
the relationships between the A variables. Specifically, if A corresponded to 
a solution of F(x) — 0, then we must have Aa;c = X^fiXxi for all monomials 
x",x^,x'^ where = x^x'^. If we added these non-linear constraints to 
the linear constraints F * A = 0, then we would essentially have the original 
polynomial system F{x) = 0. 

The system F* A = is always feasible, but the constraint Ai = 1 also 
holds for any A that corresponds to a solution x of F{x) = 0. Thus, if the 
inhomogeneous linear system F * A = 0, Ai = 1 is infeasible, then so is the 
system of polynomials F(x) — 0. 

Remark 2.1. Crucially, this linear system, F •* X = Q, Xi — 1 is 
dual to the linear system F — 1 from the previous section by Fredholm 's 
alternative meaning that F * X = 0, Xi = 1 is infeasible if and only if 
jj,^ F = 1 is feasible. 

There is a fundamental observation we wish to make here: adding 
redundant polynomial equations can lead to a tighter relaxation. 

Example 4. (Cont.) Add xif^{x) = x\ + xiX2 = to the system 
F{x) = giving the system F'{x) = where F' := {/i, /2, /sj xi/3}. The 
system F'{x) = has the same solutions as F{x) = 0. The polynomial 
equation xifs{x) = gives rise to a new linear equation X^j + A^ixg = 
giving the following linear system, F' * A = 0; 

Kl - Ai = 0, 2AxiX2 + = 0) ^xi + Xx2 = 0, Aa;2 + XxiX2 = 0' (2-2) 

The dimension of the solution space of the original system F*A = is three 
if we ignore all X variables that do not appear in the linear system, or in 
other words, if we project the solution space onto the X variables appearing 
in the system. However, the dimension of the projected solution space of 
F' * A = is two; so, F' * X = is a tighter relaxation of F{x) = 0. 
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We denote the set of solutions of the linear system F * X — as 
F° :— {X € R* : F * X — 0}, called the annihilator of F, which is a vector 
subspace of R* . The fact that adding redundant equations leads to a tighter 
linear relaxation is summarized by the following fact: For sets F ^ F ^ R, 
we have F° CF°. 

Extending this idea, consider the ideal / = {F)ii, which is the set of 
all redundant polynomials given as a polynomial combination of polynomi- 
als in F, then 1° becomes a finite dimensional vector space where dim(/°) 
is precisely the number of solutions of F(x) = over K, including mul- 
tiplicities, assuming that there are finitely many solutions. Note that by 
Hnear algebra, 1° is isomorphic to the vector space quotient R/I (see e.g., 
[48]). Furthermore, if / is radical, then dim(J°) = dim(i?//) is precisely 
the number of solutions of F{x) — 0. So, there is a direct relationship be- 
tween the number of solutions of a polynomial system and the dimension 
of the solution space of its linear relaxation (see e.g., [B]). 

Theorem 2.1. Let I C R be a zero-dimensional ideal. Then, dim(/°) 
is finite and dim{I°) is the number of solutions of polynomial system I{x) ~ 

over K including multiplicities, so \V-^{I)\ < dim(/°) with equality when 

1 is radical. 

So, if we can compute dim(/°), then we can determine the feasibility 
of I{x) = over K. Unfortunately, we cannot compute dim(/°) directly. 
Instead, under some conditions (see Theorem l2.2|) . we can compute dim(/°) 
by computing the dimension of F° when projected onto the Xx" variables 
where deg(a:") < deg(F). 

2.3. Nullstellensatz Linear Algebra Algorithm (NulLA). We 

now present an algorithm for determining whether a polynomial system of 
equations is infeasible using linear relaxations. Let F C K[a;] and again let 
F{x) = be the polynomial system f{x) = for all / G F. We wish to 
determine whether F{x) = has a solution over K. 

The idea behind NulLA [5] is straightforward: we check whether the 
linear system F>i=A = 0, Ai = lis infeasible or equivalently whether 
IS^F = 1 is feasible (i.e., 1 € (^)k) using linear algebra over K and if 
not then we add polynomials from to F and try again. We add 

polynomials in the following systematic way: for each polynomial f (z F 
and for each variable Xi, we add Xif to F. So, the NulLA algorithm is as 
follows: if * A = 0, Ai = 1 is infeasible, then F{x) = is infeasible and 
stop, otherwise for every variable Xi and every f £ F add Xif to F and 
repeat. 

In the following, we assume without loss of generality that F is closed 
under K-linear combinations, that is F = (F)k, and thus, _F is a vector 
space over K. Note that taking the closure of F under K-linear combina- 
tions does not change the set of solutions of F{x) = and does not change 
the set of solutions of F * A = 0. In practice, we must choose a vector 
space basis of F for computation, but the point we wish to make is that 
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the choice of basis is irrelevant. Moreover, we find that it is more natural 
to work with vector spaces and that it leads to a more concise exposition. 
Recall from above that F*X = 0, Ai = 1 is infeasible if and only if 1 € (-F)kj 
which when F is a vector space, simplifies to 1 € F since (-F)k = F- 

For a vector space F C R, we define F+ :— F + J^^^i ^i-^ where 
XiF := {xif : f € F}. Note that F^ is also a vector subspace of R. Then, 
F^ is precisely the linear span of F and XiF for all i = 1, . . . , n. So, the 
NulLA algorithm for vector spaces is as follows (see Algorithm [T]): if 1 e F, 
then F{x) = is infeasible and stop, otherwise set F := F+ and repeat. 
There is an upper bound on the number of times we need to repeat the 
above step given by the NuUstellensatz bound of the system F{x) — 0. 
This follows since after d iterations of NulLA, the set F contains all linear 
combinations of polynomials of the form x^f where the total degree |a| < d 
and where / was one of the initial polynomials in F. 

Algorithm 1 NulLA Algorithm [8J 

Input: A finite dimensional vector space F C R and a NuUstellensatz 
bound D. 

Output: Feasible, if F{x) — is feasible over K, else Infeasible. 
for d = 0,1,2, ...,£) do 

If 1 e F, then return Infeasible. 

F F+. 
end for 

Return Feasible. 



While theoretically the NuUstellensatz bound limits the number of 
iterations, this bound is in general too large to be practically useful (see 
[8]). Hence, in practice, NulLA is most useful for proving infeasibility (see 
Section El). 

Next, we discuss improving NulLA by adding redundant polynomials 
to F in such a way so that deg(F) does not grow unnecessarily. The im- 
proved algorithm is called the Fixed-Point NuUstellensatz Linear Algebra 
(FPNulLA) algorithm (see [7]). The basic idea behind the FPNulLA algo- 
rithm is that, if 1 ^ F, then instead of replacing F with F+ and thereby 
increasing deg(F), we check to see whether there are any new polynomials 
in F"*" with degree at most deg(F) that were not in F and add them to 
F, and then check again whether 1 ^ F. More formally, if 1 ^ F, then we 
replace F with F+ n Rd where Rd is the set of all polynomials with degree 
at most d = deg(F). We keep replacing F with F+ n Rd until either 1 S F 
or we reach a fixed point, F = F"*" n Rd- This process must terminate. 

Note that if we find that 1 g F at some stage of FPNulLA this implies 
that there exists an infeasibility certificate of the form 1 = X]i=i A/* where 
Pi,..., (3s S K[x] and the polynomials /i,...,/s £ K[x] are a vector space 
basis of the original set F. 

Moreover, we can also improve NulLA by proving that the system 
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F{x) = is feasible well before reaching the NuUstellensatz bound as fol- 
lows. When 1 ^ F and F — F+ n Rd, then we could set F := _F+ and 
d := d+l and repeat the above process. However, when we reach the fixed 
point F — n Rd, we can use the following theorem to determine if the 
system is feasible and if so how many solutions it has. First, we introduce 
some notation. Let tt^ : i?* — > Rd be the projection of a power series onto a 
polynomial of degree at most d with coefficients in K. Below, we abbreviate 
dim(7rd(F°)) as dimd(F°) and similarly dim(7rd-i(F°)) as dimd_i(F°). 

Theorem 2.2. Let F <Z R be a finite dimensional vector space and 
let d = deg(F). If F ^ F+ n Rd and dimd(F°) = dimd-i(F°), then 
dim(/°) dimd(F°) where I = {F)r. 

See [351 (3 for a proof of Theorem 12.21 Recall from Theorem 12.11 that 
there are dim(J°) solutions of F{x) = over K including multiplicities 
where / = (F) r and exactly dim(/°) solutions when / is radical. 

There are many equivalent forms of the above theorem that appear 
in the literature, (see e.g., [351 1131 113 )• Note that the condition that 
F = F+r]Rdis equivalent to the condition that Aiu\d{F°) = dimc;((F+)°)). 
Also, since Rd is a vector space and F C Rd is vector subspace, we can 
form the vector space quotient Rd/F, which is isomorphic to 7rd(_F°) (see 
for example [48]), and thus, dimf;(i^°) = dim{Rd/F) = dim{Rd) - dim(F) 
where dim(i?d) = ("+''). Similarly, dim(i?d-i/i^) = dimd_i(F°) and 
dim{Rd/ F^) — dimd{{F^)°). Thus, in practice, checking the conditions of 
Theorem 12 .21 means computing dim(F), dim(Fni?d_i) and dim(F+ni?d). 

We can now present the FPNulLA algorithm [36l [7]. See [36] for a 
proof of termination. 



Algorithm 2 FPNulLA Algorithm 
Input: A vector space F c R. 

Output: The number of solutions of F{x) ~ over K up to multiplicities. 

Let d = deg(F). 
loop 

if 1 e F then Return 0. 
while F 7^ F+ n i?d do 

Set F F+ n Rd- 

if 1 S F then return 0. 
end while 

if dimd(F°) = dimrf_i(F°) then return dim<i(F°). 
F := F+. 
d:^d+l. 
end loop 



Example 5. Consider the following feasible system with polynomials 
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in K[x] with IK = F2. 

l + x + x^ ^Q, 1 + y + = 0, + xy + ^Q. 

This system has two solutions over K = F2. Let F :— (l + .T + x^,l + y + 
y'^,x^ +xy + y^)K- Then, 1^ F and deg{F) = 2. Now, 

F+ = F + xF + yF 

= F + (a; + X X ,x ^ xy ^ xy , x + a; y + xy )k 
+ {y + xy + x'^y, y + y'^ + y^, x^y + xy"^ + y^)K 

Then, F+ D R2 = {I + x + x"^ ,1 + y + y"^ .x"^ + xy + y"^ ,1 + x + y)K- So, 
F ^ F+ nR2. Next, let F := F+ R2. Then, F ^ F+ D R2. Moreover, 
dim2(-F°) 2 and dimi(_F°) = 2. Therefore, the system is feasible. 

2.4. Experimental results. In this section, we summarize experi- 
mental results for graph 3-coloring from |7j, which illustrate the practical 
performance of the NulLA and FPNulLA algorithms. For further and more 
detailed results, see [8l[34l[7]. Experimentally, for graph 3-coloring, NulLA 
and FPNulLA are well-suited to proving infeasibility, that is, that no 3- 
coloring exists. The polynomials encoding of 3-coloring that is used here 
is over F2 (see Proposition II. ip and thus any linear algebra operations are 
very fast. However, even though in theory NulLA and FPNulLA can deter- 
mine feasibility, for the experiments described below NulLA and FPNulLA 
were not able to prove feasibility in practice. 

We refer to the number of iterations that NulLA takes to solve a given 
system of equations as the NulLA degree of the system. Similarly to the 
NulLA degree, we refer to the number of outer iterations that FPNulLA 
takes to the system as the FPNulLA degree of the system. We can consider 
the NulLA degree and the FPNulLA degree as measures of the hardness of 
proving infeasibility of the system. In this section, we present experimen- 
tal evidence that the NulLA degree of an infeasible combinatorial system 
is a good measure of the hardness of proving infeasibility of the system. 
Similarly, we present experimental evidence (see also ^ for theoretical ev- 
idence) suggesting that the FPNulLA degree is also a good measure of the 
hardness of a problem and an even better measure than the NulLA degree. 

Here, we are interested in the percentage of randomly generated graphs 
whose polynomial system encoding has a NulLA degree of one or a FP- 
NulLA degree of one. The G{n,p) model [T3j is used for generating random 
graphs where n is the number of vertices and p is the probability that an 
edge is included between any two vertices. Also, without loss of generality, 
for a slightly smaller polynomial encoding, the color of one of the vertices 
of each randomly generated graph was fixed. 

The experimental results are presented in Figure [T] (taken from |7j), 
which plots the percentage of 1000 random graphs in G(100,p) that were 
proven infeasible with a NulLA degree of one, with a FPNulLA degree of 



POLYNOMIALS IN COMBINATORIAL OPTIMIZATION 



15 



one, or with an exact method versus the p value. The exact method used 
was to model graph 3-coloring as a Boolean satisfiability problem [T^] and 
then use the program zchaf f [5D] to solve the satisfiability problem. 




p - edge probability 

Fig. 1. N on- 3- colorable graphs with NulLA or FPNulLA degree of 1 

It is well-known that there is a distinct phase transition from feasibil- 
ity to infeasibility for graph 3-coloring, and it is at this phase transition 
that graphs exists for which it is difficult on average to prove infeasibility or 
feasibility (see [l9j). Observe that the infeasibility curve for NulLA resem- 
bles that of the exact infeasibility curve and that the infeasibility curve for 
FPNulLA also resembles the infeasibility curve and clearly dominates the 
infeasibility curve for NulLA. These results support that statement that 
the NulLA degree or FPNulLA degree is a reasonable measure of the hard- 
ness of proving infeasibility since those graphs that require a higher degree 
than one are located near the phase transition. 

2.5. Application: the structure of non-3-colorable graphs. For 

a given class of combinatorial system of equations, it is of interest to under- 
stand the growth of the NulLA degree or FPNulLA degree. For some fixed 
degree, it is also interesting to characterize which graphs can be proved 
at that degree to lack a certain property. In this section, we state a com- 
binatorial characterization of those graphs whose combinatorial system of 
equations encoding 3-colorability has a NulLA degree of one and recall 
bounds for the NulLA degree (see [34]): 

Theorem 2.3. The NulLA degree for a polynomial encoding over 
¥2 of the 3-colorability of a graph with n vertices with no 3-coloring is at 
least one and at most 2n. Moreover, in the case of a non- 3- colorable graph 
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containing an odd-wheel or a J^-clique as a subgraph, the NulLA degree is 
exactly one. 

Now we look at those non-3-colorable graphs that have a degree one 
NulLA degree. Let A denote the set of all possible directed edges or arcs in 
the graph G. We are interested in two types of substructures of the graph 
G: oriented partial-3-cycles and oriented chordless 4-cycles (see Figure [^75|) . 
An oriented partial-3-cycle is a set of two arcs of a 3-cycle, that is, a 
set {(i, j), (j, /c)} also denoted {i,j,k) where {i,j),{j,k),{k,i) e A. An 
oriented chordless 4-cycle is a set of four arcs {{i,j), {j,l), {l,k), {k,i)} 
also denoted k, I) where (i, j), {j, I), {I, fc), (fc, i) £ A and {j, k), {i, I) ^ 
A. 

i k i k 

Fig. 2. (i) oriented partial 3-cycle and (ii) an oriented chordless 4-cycle 

Now, we can state a sufficient condition for non-3-colorability . This 
sufficient condition is satisfied if and only if the combinatorial system en- 
coding 3-coloring has a NulLA degree of one, which is proved in j7]. 

Theorem 2.4. The graph G is not 3-colorable if there exists a set C 
of oriented partial 3-cycles and oriented chordless 4-cycles such that 
1. |C(i j)| + |C(ji)| = (mod 2) for all {i,j) e E and 

2- E(-,,)eA,.<,l%.)l^l (mod 2) 
where |C(i j) | denotes the number of cycles inC ( either 3-cycles or ^-cycles) 
in which the arc (i,j) € A appears. 

Condition 1 in Lemma 12.41 means that every undirected edge of G is 
covered by an even number of directed edges from cycles in G (ignoring 
orientation). Condition 2 in Lemma 12.41 means that, given any orientation 
of G, the total number of times the arcs in that orientation appear in the 
cycles of C is odd. The particular orientation we use in Lemma [2.41 is the 
orientation given by the set of arcs {(?, j) € A : i < j}, but the particular 
orientation we use for Condition 2 is irrelevant (see [7J). 

Example 6. Consider the Grotzsch graph (Mycielski 4) in Figure\3i 
which has no 3-coloring. It contains no 3-cycles. Now, consider the follow- 
ing set of oriented chordless 4-cycles, which we show gives a certificate of 
non-3-colorability by Lemma \2.4\ 

G := {(1, 2, 3, 7), (2, 3, 4, 8), (3, 4, 5, 9), (4, 5, 1, 10), (1, 10, 11, 7), 
(2, 6, 11, 8), (3, 7, 11, 9), (4, 8, 11, 10), (5, 9, 11, 6)}. 

Figure\^ illustrates the edge directions for the 4-cycles of G . Each undi- 
rected edge of the graph is contained in exactly two 4-cycles, so G satisfies 
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Condition 1 of Lemma \2.4\ Now, 

|C(6,11)I = |C(7,11)I = |C(8,11)I = |C(9,11)I = |C(1041)I = 1' 

and |C(i j) = (mod 2) for all other arcs {i,j) G A where i < j. Thus, 
J2 = l (mod 2), 

{i,j)eAa<j 

SO Condition 2 is satisfied, and therefore, the graph has no 3- coloring. 

1 




Fig. 3. Grotzsch graph 



3. Adding polynomial inequalities. Up until this point we have 
worked over arbitrary fields (with special attention to finite fields due to 
their fast and exact computation), where the only allowable constraints 
were equations. Now we turn our attention to the real case (i.e. K = M), 
where we have the additional possibility of specifying inequalities (more 
generally, one can work over ordered or formally real fields). In this case, 
following the terminology of real algebraic geometry, we call the solution set 
of a system of polynomial equations and inequalities a basic semialgehraic 
set. Note that convex polyhedra correspond to the particular case where 
all the constraint polynomials have degree one. As we have seen earlier 
in the Positivstellensatz (Theorem 11.41 above) , the emptiness of a basic 
semialgebraic set can be certified through an algebraic identity involving 
sum of squares of polynomials. 

The connection between sum of squares decompositions of polynomi- 
als and convex optimization can be traced back to the work of N. Z. Shor 
|46| . His work went relatively unnoticed for several years, until several au- 
thors, including Lasserre, Nesterov, and Parrilo, observed around the year 
2000 that the existence of sum of squares decompositions and the search 
for infeasibility certificates for a semialgebraic set can be addressed via a 



18 



J.A. DE LOERA, P.N. MALKIN AND P.A. PARRILO 



sequence of semidefinite programs relaxations [231 ISSl SOI HE] • The first 
part of this section will be a short description of the connections between 
sums of squares and semidefinite programming, and how the Positivstel- 
lensatz allows, in a analogous way to what was presented in Section [2] for 
the NuUstellensatz, for a systematic way to formulate these semidefinite 
relaxations. 

A very central preoccupation of combinatorial optimizers has been 
the understanding of the facets that describe the integer hull (normally 
binary) of a combinatorial problem. As we will see in the last part of 
this survey, one can recover quite a bit of information about the integer 
hull of combinatorial problems from a sequence combinatorially controlled 
SDPs. This kind of approach was pioneered in the lift-and-project method 
of Balas, Ceria and Cornuejols IJ, the matrix-cut method of Lovasz and 
Schrijver [33] and the linearization technique of Sherali- Adams [ISj. Here 
we try to present more recent developments (see [29) and references therein 
for a very extensive survey). 

3.1. Sums of squares, SDP, and feasibility of semialgebraic 
sets. A multivariate polynomial p{x) is a sum of squares (SOS for short) 
if it can be written as a sum of squares of other polynomials, i.e., 

p{x) =Y,<lKx), g,(x)eM[x]. 

i 

If p{x) is SOS, then clearly p{x) > for all x e M". 

Example 7. The polynomial p{xi, X2) = x\ — X1X2 + X2 + 1 is SOS. 
Among infinitely many others, it has the following decompositions: 

3 1 

pixi,X2) = -(Xi - xl)'^ + -{Xi +xiy + 1 

= i(3 - xir + \xl + ^(9X1 - 160:^)^ + 

The sum of squares condition is a quite natural sufficient test for polyno- 
mial non-negativity. Thus instead of asking whether even degree polyno- 
mials are non-negative we ask the easier question whether they are sums 
of squares. More importantly, as we shall see, the existence of a sum of 
squares decomposition can be decided via semidefinite programming. 

Theorem 3.1. A polynomial p{x) is SOS if and only if p{x) — z^Qz, 
where z is a vector of monomials in the Xi variables, and Q is a symmetric 
positive semidefinite matrix. 

By the theorem above, every SOS polynomial can be written as a 
quadratic form in a set of monomials, with the corresponding matrix being 
positive semidefinite. The vector of monomials z in general depends on 
the degree and sparsity pattern oi p{x). If p{x) has n variables and total 
degree 2d, then z can always be chosen as a subset of the set of monomials 
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of degree less than or equal to d, which has cardinality ("^'^) ■ 

Example 8. Consider again the polynomial from Example^ It has 
the representation 



p{xi,X2) = 





1 


T 


6 





-2 


" 




1 


1 









4 










X2 


6 






-2 





6 


-3 




X2 




Xl 










-3 


6 




Xl 



and the matrix in the expression above is positive semidefinite. 

In the representation f{x) = z'^Qz, for the right- and left-hand sides 
to be identical, all the coefficients of the corresponding polynomials should 
be equal. Since Q is simultaneously constrained by linear equations and a 
positive semidefiniteness condition, the problem can be easily seen to be 
directly equivalent to an semidefinite programming feasibility problem in 
the standard primal form. 

Now we describe an algorithm, and illustrate it with an example, on 
how we can use SDPs to decide the feasibility of a system of polynomial 
inequalities. Exactly as we did for the NuUstellensatz case, we can look for 
the existence of a Positivstellensatz certificate of bounded degree D. Once 
we assume that the degree D is fixed we can apply Theorem l3.1l and obtain 
a reformulation as a semidefinite programming problem. We formalize this 
description in the following algorithm: 



Algorithm 3 Bounded degree Positivstellensatz [39, 40 
Input: A polynomial system {fi{x) = 0,gi{x) > 0} and a Positivstellen- 
satz bound D. 

Output: Feasible, if {fi{x) — 0,gi{x) > 0} is feasible over R, else In- 
feasible. 

for d = 0,1,2, ...,£) do 

If there exist f3„ s^ & such that -1 = J2i Pifi + EaG{o,i}" ^"ff"' 
with Sa SOS, deg{l3ifi) < d, deg(sQg") < d then return Infeasible. 

d:=d+l. 
end for 

Return Feasible. 



Notice that the membership test in the main loop of the algorithm 
is, by the results described at the beginning of this section, equivalent to 
a finite-sized semidefinite program. Similarly to the NuUstellensatz case, 
the number of iterations (i.e., the degree of the certificates) serves as a 
quantitative measure of the hardness in proving infeasibility of the system. 
As we will describe in more detail in Section 13.41 in several situations one 
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can give further refined characterization on these degrees. 

Example 9. Consider the polynomial system {f = 0, g > 0}, where 

/ := .T2 + Xi + 2 = 0, g := xi-xj+i > 0. 

By the Positivstellensatz, there are no solutions {xi,X2) G if and only 
if there exist polynomials ti, si, S2 G R[a;i, X2] that satisfy 

si + S2 ■ g + ti ■ f = — f , where si and S2 are SOS. (3.f ) 

At the D-th SDP relaxation of the polynomial problem {f — 0, g > 
0}, one asks whether there exists a solution {ti, 81,82) to iS. 1]) where the 
polynomial si has degree < D and the polynomials 52,^1 have degree < 
D — 2. For each fixed positive integer D this can be tested by a (possibly 
large) semidefinite program. Solving this for D = 2, we find the infeasibility 
certificate 

si = i+2(.T2 + |)^ + 6(a;i-i)', S2 = 2, ti = -6. 

The resulting identity iS.l]) proves the inconsistency of the system. 

As outhned in the preceding paragraphs, there is a direct connec- 
tion going from general polynomial optimization problems to SDP, via the 
Positivstellensatz infeasibility certificates. Even though we have discussed 
only feasibility problems here, there are obvious straightforward connec- 
tions with optimization. For instance, by considering the emptiness of the 
sublevel sets of the objective function, or using representation theorems 
for positive polynomials, sequences of converging bounds indexed by cer- 
tificate degree can be directly constructed; see e.g. [39l [23l E]. These 
schemes have been implemented in software packages such as SOSTOOLS 
[15], GloptiPoly 113, and YALMIP [3D]. 

3.2. Semidefinite programming relaxations. In the last section, 
we have described the search for Positivstellensatz infeasibility certificates 
formulated as a semidefinite programming problem. We now describe an al- 
ternative interpretation, obtained by dualizing the corresponding semidefi- 
nite programs. This is the exact analogue of the construction presented in 
Section 12.21 ^^^d is closely related to the approach via truncated moment 
sequences developed by Lasserre p3|. 

Recall that in the approach in Section 12.21 the linear relaxations were 
constructed by replacing every monomial x" by a new variable A^a . Fur- 
thermore, new redundant equations were obtained by multiplying an exist- 
ing constraint f{x) = by terms of the form Xi, yielding Xif{x) = (essen- 
tially, generating the ideal of valid equations). In the inequality case, and as 
suggested by the Positivstellensatz, new inequality constraints will be gen- 
erated by both squarefree multiplication of the original constraints, and by 
multiplication against sums of squares. That is, if ^^(a;) > and gj{x) > 
are valid inequalities, then so are gi{x)gj{x) > and gi{x)s{x) > 0, where 
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s{x) is SOS. After substitution with the extended variables A, we then ob- 
tain a new system of hnear equations and inequahties, with the property 
that the resulting inequality conditions are semidefinite conditions. The 
presence of the semidefinite constraints arises because we do not specify a 
priori what the multipliers s{x) are, but only give their linear span. 

Example 10. Consider the problem discussed earlier in Example\^ 
The corresponding relaxation is (for D = 2): 

Ai \x-^ 

^Xi Aj.2 

plus the condition Ai > (without loss of generality, we can take — 
This is a semidefinite programming problem, and in this case, its infeasi- 
bility directly shows that the original system of polynomial inequalities does 
not have a solution. 

An appealing geometric interpretation follows from considering the 
projection of the feasible set of these relaxations in the space of original 
variables (i.e., Aj..). For the linear algebra relaxations of Section [221 we 
obtain outer approximations to the affine hull of the solution set (an al- 
gebraic variety), while the SDP relaxation described here constructs outer 
approximations to the convex hull of the corresponding semialgebraic set. 
This latter viewpoint will be further discussed in Section [3731 for the case 
of equations arising from combinatorial problems. 

3.3. Theta bodies. Recall that traditional modeling of combinato- 
rial optimization problems often uses 0/1 incidence vectors. The set S of 
solutions of a combinatorial problem (e.g., the stable sets, traveling sales- 
man tours) is often computed through the (implicit) convex hull of such 
incidence vectors. Just as in the stable set and max-cut examples in Propo- 
sition [Ol the incidence vectors can be seen at the set of real solutions to a 
system of polynomial equations: fi{x) = f2(x) = ■ ■ ■ = fm(x) = 0, where 
fi7 ■ ■ ■ , fm G '■— M.[xi, . . . , Xn]- Over the years there have been well- 
known attempts to understand the structure of these convex hulls through 
semidefinite programming relaxations (see [45l [33l [25l [32] ) and in fact they 
are closely related [26l [29] . Here we wish to summarize some recent results 
that give appealing structural properties, in terms of the associated system 
of equations (see [TUlHj for details). 

Let us start with a historically important example: Given an undi- 
rected finite graph G = (V, E), consider the set Sq of characteristic vectors 
of stable sets of G. The convex hull of Sg, denoted by STAB(G'), is the sta- 
ble set polytope. As we mentioned already the vanishing ideal of Sq is given 
by Ig '■= {xf — Xi (y i ^ V), XiXj (V {i,j} € E)) which is a real radical 
zero-dimensional ideal in M.[x]. In [31j . Lovasz introduced a semidefinite 
relaxation, TH(G'), of the polytope STAB(G'), called the theta body of G. 
There are multiple descriptions of TH(G'), but the one in [331 Lemma 2.17], 



'^XiX2 
A^2 



^0, 



Aa:2-l~A^2+2Ai — 0, Axi ^A2,2 +3Ai > 0, 
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for instance, shows that TH(G) can be defined completely in terms of the 
polynomial system Iq. It is easy to show that STAB(G) C TH(G'), and 
remarkably, we have that STAB(G) = TH(G) if and only if the graph is 
perfect. We will now explain how the case of stable sets can be generalized 
to construct theta bodies for many other combinatorial problems. 

We will construct an approximation of the convex hull of a finite set of 
points S, denoted conv(S'), by a sequence of convex bodies recovered from 
"degree truncations" of the defining polynomial systems. In what follows 
I will be a radical polynomial ideal. A polynomial / is non-negative 
modulo I, written as / > mod 7, if f{s) > for all s G Vr(/). More 
strongly, the polynomial / is a sum of squares (sos) mod / if there 
exists hj e M.[x] such that / = h'j mod / for some t, or equivalently, 

/ — hj G I. If, in addition, each hj has degree at most k, then we 

say that / is fc-sos mod /. The ideal / is fc-sos if every polynomial that 
is non-negative mod I is fc-sos mod I. If every polynomial of degree at 
most d that is non-negative mod I is k-sos mod I, we say that / is {d, k)- 
sos. We say that a polynomial ideal / C M.[x\ is TH^-exact if every linear 
polynomial that is non- negative over Vr(/), the real variety of I, is a sum 
of squares of polynomials of degree at most k modulo /. 

Note that conv(VK(/)), the convex hull of Va(/), is described by the 
linear polynomials / such that / > mod /. A certificate for the non- 
negativity of / mod I is the existence of a sos-polynomial X]5=i hj that 
is congruent to / mod /. One can now investigate the convex hull of S 
through the hierarchy of nested closed convex sets defined by the semidef- 
inite programming relaxations of the set of (d, fc)-sos polynomials. 

Definition 3.1. Let I C Wyx] he an ideal, and let k be a positive 
integer. Let c R[x] be the set of all polynomials that are fc-sos mod I. 

1. The k-th theta body of I is 

THfc(/) := {a; e M" : f{x) > for every linear / e Sfe}. 

2. The ideal I is THfe-exact if the k-th theta body THfc(J) coincides 
with the closure o/ conv(VR(/)). 

3. The theta- rank of I is the smallest k such that THfc (7) coincides 
with the closure o/conv(VK(7))- 

Example 11. Consider the ideal I = {x^y - 1) C K[a;, y]. Then 
conv(VR(/)) = {{pijP2) G '■ P2 > 0}, and any linear polynomial that 
is non-negaMve over Vr(/) is of the form a + Py, where a,/3>0. Since 
ay + f3 = {^/axyY + (v^)^ "^orf I, I is (1, 2)-sos and TB.2-exact. 

Example 12. For the case of the stable sets of a graph G, one can 
see that 



THi (Jg) 



ye 



3 M ^ 0, M e IR(»+i)x(n+i) such that 
Moo = 1, 

Moi ^ Mu^yi^ iGV 

Mij=Q^ {i,j}GE 
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It is known that THi(/g) is precisely Lovasz's theta body of G. The ideal 
Ic is THi-exact precisely when the graph G is perfect. 

By definition, THi(/) D TH2(/) 3 ••• 3 conv(VE(/)). As seen in 
Example [TTl conv(VR(/)) may not always be closed and so the theta-body 
sequence of / can converge, if at all, only to the closure of conv(VR(/)). 
But the good news for combinatorial optimization is that there is plenty of 
good behavior for problems arising with a finite set of possible solutions. 

3.4. Application: cuts and exact finite sets. We discuss now a 
few important combinatorial examples. As we have seen in Section [2.51 for 
3-colorability, and in the preceding section for stable sets, in some special 
cases it is possible to give nice combinatorial characterizations of when 
low-degree certificates can exactly recognize infeasibility. Here are a few 
additional results for the real case: 

Example 13. For the max- cut problem we saw earlier, the defining 
vanishing ideal is I{SG) — {x^ ^ x^ V e £ E, x^V T an odd cycle in G) . 
In this case one can prove that the ideal I{SG) is THi-exact if and only 
if G is a bipartite graph. In general the theta-rank of I{SG) is bounded 
above by the size of the max- cut in G. There is no constant k such that 
THfc(/(S'G')) — conv(S'G), for all graphs G. Other formulations of max- cut 
are studied in [14j . 

Recall that when S C K" is a finite set, its vanishing ideal I{S) is 
zero-dimensional and real radical. In what follows, we say that a finite set 
S C K" is exact if its vanishing ideal I{S) C R[x] is THi-exact. 

Theorem 3.2 ([15J). For a finite set S* C R", the following are 
equivalent. 

1. S is exact. 

2. There is a finite linear inequality description of conv(5) in which 
for every inequality g{x) > 0, g is 1-sos mod I{S). 

3. There is a finite linear inequality description of conv(S') such that 
for every inequality g{x) > 0, every point in S lies either on the 
hyperplane g{x) = or on a unique parallel translate of it. 

4- The polytope conv(S') is affinely equivalent to a compressed lattice 
polytope ( every reverse lexicographic triangulation of the polytope 
is unimodular with respect to the defining lattice). 
Example 14. The vertices of the following 0/1-polytopes in R" are 
exact for every n: (1) hypercubes, (2) (regular) cross polytopes, (3) hyper- 
simplices (includes simplices), (4) joins of 2-level polytopes, and (5) stable 
set polytopes of perfect graphs on n vertices. 
More strongly one can say the following. 

Proposition 3.1. Suppose S CR^ is a finite point set such that for 
each facet F of cohy(S) there is a hyperplane Hp such that Hpriconv{S) = 
F and S is contained in at most t-\-l parallel translates of Hp. Then I{S) 
is TKt-exact. 

In [15] the authors show that theta bodies can be computed explicitly 
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as projections to the feasible set of a semidefinite program. These SDPs 
are constructed using the combinatorial moment matrices introduced by 

m- 

4. Recovering solutions in the feasible case. In principle, it is 
possible to find the actual roots of the system of equations (and thus the 
colorings, stable sets, or desired combinatorial object) whenever the relax- 
ations are feasible and a few additional conditions are satisfied. Here we 
discuss mostly the linear algebra relaxations case, but the semidefinite case 
is very similar; see e.g. [HI [24] for this case. 

We describe below how, under certain conditions, it is possible to re- 
cover the solution of the original polynomial system from the relaxations 
(linear or semidefinite) described in earlier sections. The main concepts 
are very similar for both methodologies, and are based on the well-known 
eigenvalue methods for polynomial equations; see e.g. [SI §2.4]. The key 
idea for extracting solutions is the fact that from the relaxations one can 
obtain a finite-dimensional representation of the vector space R/I and its 
multiplicative structure, where I is the ideal (i^)_R (in the case of linear 
relaxations). In order to do this, we need to compute a basis of the vec- 
tor space R/I, and construct matrix representations for the multiplication 
operators Mx^ '■ f f—^ xtf. Then, we can use the eigenvalue/eigenvector 
methods to compute solutions (see e.g., [lO]). 

A sufficient condition for the existence of a suitable basis of R/I is 
given by Theorem 12.21 Under this condition, multiplication matrices M^^ 
can be easily computed. In particular, if we have computed a set F d R 
that satisfies the conditions of Theorem 12.21 by running FPNulLA, then 
finding a basis oi R/I and computing its multiplicative structure is straight- 
forward using linear algebra (see e.g., j36j ) . By construction, the matrices 

commute pairwise, and to obtain the roots one must diagonalize the 
corresponding commutative algebra. It is well-known (see, e.g., ^), that 
this can be achieved by forming a random linear combination of these ma- 
trices. This random matrix will generically have distinct eigenvalues, and 
the corresponding matrix of eigenvectors will give the needed change of 
basis. In the case of a finite field, it is enough to choose the random co- 
efficients over an algebraic extension of sufficiently large degree, instead of 
working over the algebraic closure (alternatively, the more efficient meth- 
ods in pTj can be used). The entries of the diagonalized matrices directly 
provide the coordinates of the roots. 

Remark 4.1. The condition in Theorem 112. 2\) can in general he a 
strong requirement for recovery of solutions, since it implies that we can 
obtain all solutions of the polynomial system. In some occasions, it may be 
desirable to obtain just a single solution, in which case weaker conditions 
may be of interest. 

Example 15. Consider the following polynomial system over ¥2, that 
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Fig. 4. Graph for Examvle\15\ 



corresponds to the 3-colorings of the six-node graph in Figure^ 



1 = \/ieV, 



We add to these equations the symmetry-breaking constraint Xq — \ . After 
running NuLLA with this system as an input, we obtain multiplication 
matrices over ¥2, of dimensions 4x4, given by: 
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Diagonalizing the corresponding commutative algebra, we obtain the change 
of basis matrix given by 



1 


1 


1 


1 




LO 


LO 




1 


1 




UJ 


0.2 


LO 


1 


1 



where lo is a primitive root of 1, i.e., it satisfies + w + 1 = 0. It can be 
easily verified that all the matrices T^^Mx^T are diagonal, and given by: 

T-^Mx,T = diag[w, Lo^, UJ, uj^] T-^M^.T = dmg[u;^, lo, 1, 1] 
T-^Mx,T = diag[l, 1, uj^ uj] T-^Mx,T = diag[w, co^ uj\ uo] 

which correspond to the four possible 3-colorings of the graph. For instance, 
from the second diagonal entry of each matrix we obtain the feasible coloring 

{X(i,Xi,X2,X3„X4:,X^) {l,L0'^,U}, l,Lo'^,Lo). 
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